TRACES OF HIGH POWERS OF THE FROBENIUS CLASS 
IN THE HYPERELLIPTIC ENSEMBLE 



^ ■ ZEEV RUDNICK 

o 

. Abstract. The zeta function of a curve over a finite field may be ex- 

C^' pressed in terms of tlie cliaracteristic polynomial of a unitary symplectic 

matrix O, called the Frobenius class of the curve. We compute the ex- 
pected value of tr(0") for an ensemble of hyperelliptic curves of genus 
g over a fixed finite field in the limit of large genus, and compare the 
results to the corresponding averages over the unitary symplectic group 
^ . USp(25). We are able to compute the averages for powers n almost up to 

^yr ' 4g, finding agreement with the Random Matrix results except for small 

• , n and for n = 2g. As an application we compute the one-level density 

1^ ' of zeros of the zeta function of the curves, including lower-order terms, 

d . for test functions whose Fourier transform is supported in (—2,2). The 

results confirm in part a conjecture of Katz and Sarnak, that to leading 
order the low-lying zeros for this ensemble have symplectic statistics. 

> 

\o ■ 

^ . 1. Introduction 

Fix a finite field of odd cardinality, and let C be a non singular projec- 
^ ■ five curve defined over Fg. For each extension field of degree n of Fg, denote 

' by Nn{C) the number of points of C in F^n. The zeta function associated 

■ to C is defined as 



Nn{C) — , \u\ < 1/q 

n 



n 

n=l 



and is known to be a rational function of u of the form 

(1 1) Zc{u) = -PgH 

(1 — — qu) 
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where Pc{u) is a polynomial of degree 2g with integer coefficients, satisfying 
a functional equation 

Pc{u) = (quyPci-) . 

qu 

The Riemann Hypothesis, proved by Weil [19], is that the zeros of P{u) all 
lie on the circle |m| = 1/^^- Thus one may give a spectral interpretation of 
Pc{u) as the characteristic polynomial of a 2(7 x 2g unitary matrix Be 

Pciu) = det(/ - u^Qc) 

so that the eigenvalues e*^^ of Oc correspond to zeros q~^l'^e~^^^ of Zc{u). 
The matrix (or rather the conjugacy class) Be is called the unitarized Frobe- 
nius class of C . 

We would like to study the how the Frobenius classes 9c change as we 
vary the curve over a family of hyperelliptic curves of genus g, in the limit of 
large genus and fixed constant field. The particular family TC2g+i we choose 
is the family of all curves given in affine form by an equation 

CQ:y' = Q{x) 

where 

Q(x) = x^3+^ + a2g + ■ ■ ■ + ao G Fjx] 

is a squarefree, monic polynomial of degree 2g + 1. The curve Cq is thus 
nonsingular and of genus g. 

We consider TC2g+i as a probability space (ensemble) with the uniform 
probability measure, so that the expected value of any function F on H2g+i 
is defined as 

Katz and Sarnak showed that as g oo, the Frobenius classes 6q 
become equidistributed in the unitary symplectic group USp(25f) (in genus 
one this is due to Birch [2] for q prime, and to Deligne [3]). That is for any 
continuous function on the space of conjugacy classes of USp(25f), 



lim {F{Qq)) = I F{U)dU 

1^°° -'USp{2g) 
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This implies that various statistics of the eigenvalues can, in this limit, be 
computed by integrating the corresponding quantities over USp(25f). 

Our goal is to explore the opposite limit, that of fixed constant field and 
large genus (g fixed, g oo), cf [HIE]. Since the matrices Qq now inhabit 
different spaces as g grows, it is not clear how to formulate an equidistribu- 
tion problem. However one can still meaningfully discuss various statistics, 
the most fundamental being various products of traces of powers of Qq, 
that is (jYj=i^^i^Q)'^ ■ Here we study the basic case of the expected val- 
ues (trBg) where n is of order of the genus g. 

The mean value of traces of powers when averaged over the unitary sym- 
plectic group USp(25f) are easily seen to be [5] 



;i.2) 



where 



'29 



tr(t/")cit/ 



USp(2g) 



n = 

1 < 1^1 < 2^ 
\n\ > 2g 



n even 
n odd 



We will show: 



Theorem 1. For all n > we have 

-r]n, <n <2g 



-1-^, n = 2g 



- y 



deg P 



qii/^ ^ |P| + 1 
p prime 



0, n> 2g 

the sum over all irreducible monic polynomials P, and where \P\ ■= q'^°^^ . 

In particular we have 
Corollary 2.7/3 log^ g < n < Ag — 5 log^ g but n ^ 2g then 



tr W'dU + oi-] 

USp(2<?) 9' 
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We do however get deviations from the Random Matrix Theory results 
(11.21) for small values of n, for instance 



JUSp(2g) Q + 

and for n = 2g where we have 



(treg')^ / 



tr U^^dU ^ 



'USp{23) <? - 1 

Analogous results can be derived for mean values of products, e.g. for 
(tr 9q tr Bq) , when m + n < Ag, see § [61 

To prove these results, we cannot use the powerful equidistribution theo- 
rem of Deligne [3], as was done for the fixed genus case in [TT]. Rather, we 
use a variant of the analytic methods developed to deal with such problems 
in the number field setting [T51 HUl HB] • Extending the range of our results 
to cover n > 4(7 is a challenge. 

1.1. Application: The one-level density. The traces of powers deter- 
mine all linear statistics, such as the number of angles 6j lying in a subin- 
terval of ]R/27rZ, or the one- level density, a smooth linear statistic. To 
define the one- level density, we start with an even test function /, say in 
the Schwartz space iS(]R), and for any > 1 set 

F{e) :=E/(^(|:-^)) 

k£Z 

which has period 27r and is localized in an interval of size ~ 1/A^ in ]R/27rZ. 
For a unitary N x N matrix U with eigenvalues e*^J , j = 1, . . . A^, define 

N 

i=i 

which counts the number of "low-lying" eigenphases 9j in the smooth inter- 
val of length 1/N around the origin defined by /. 

Katz and Sarnak conjectured [9j that for fixed g, the expected value of 
Zf over H2g+i will converge to /usp(2g) Zf{U)dU as g —>■ oo for any such test 
function /. Theorem [1] implies: 
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Corollary 3. // / G S(M.) is even, with Fourier transform f supported in 
(-2,2) then 

JvSp(2g) 9 9 

where 

dev{f) = m ^f^-/(l^ 



|P|2 - 1 ' ' 'q-l 
p prime 

the sum over all irreducible monic polynomials P. 

To show corollary [3l one uses a Fourier expansion to see that 

/(x)rfx + -^/(-)tr[/V 

Averaging Zf{U) over the symplectic group USp(25f), using (11.20 . and as- 
suming / is even, givei^ 

/ Zf{u)du = m-- J2 /(-) 

and then we use Theorem [1] to deduce Corollary [31 

Corollary [3] is completely analogous to what is known in the number 
field setting for the corresponding case of zeta functions of quadratic fields, 
except for the lower order term which is different: While the coefficient of 
/(O) is as in the number field setting [H], the coefficient of /(I) is special 
to our function-field setting. 

1.2. Acknowledgments. We thank Par Kurlberg and Nicolas Templier 
for their comments. Supported by the Israel Science Foundation (grant 
No. 925/06) and by the Oswald Veblen Fund at the Institute for Advanced 
Study, Princeton. 

2. Quadratic L-functions 

In this section we give some known background on the zeta function of 
hyperelliptic curves. The theory was initiated by E. Artin [1]. We use Rosen 
[T6] as a general reference. 



iNote that as <? ^ oo, J^^^^^^^ Zf{U)dU ^ f{x) (l 



sin 27TX \ 



dx 
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2.1. For a nonzero polynomial / G Fg[a;], we define the norm |/| := q'^'^^^. 
A "prime" polynomial is a monic irreducible polynomial. For a monic poly- 
nomial /, The von Mangoldt function A(/) is defined to be zero unless 
/ = P'^ is a prime power in which case A{P'') = degP. 

The analogue of Riemann's zeta function is 

p prime 

which is shown to equal 

(2-1) C,(^) = 

Let vrg(n) be the number of prime polynomials of degree n. The Prime 
Polynomial Theorem in ¥q[x] asserts that 

n 

n 

which follows from the identity (equivalent to (12. ip ) 
(2.2) ^(/) = 9" 

deg(/)=n 

the sum over all monic polynomials of degree n. 

2.2. For a monic polynomial D G ¥q[x] of positive degree, which is not a 
perfect square, we define the quadratic character xd in terms of the qua- 
dratic residue symbol for Fg[a;] by 

Xnif) = (^j 



and the corresponding L-function 

p 

the product over all monic irreducible (prime) polynomials P. Expanding 
in additive form using unique factorization, we write 



, , 1 
\u\ < - 



/:(m,xd) = 5^^d(/5)m' 



/3 

P>0 
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with 



deg B=i3 

B monic 

If D is non-square of positive degree, then Ad{P) = for /3 > degD and 
hence the L-function is in fact a polynomial of degree at most degD — 1. 

2.3. To proceed further, assume that D is square-free (and monic of pos- 
itive degree). Then C{u,xd) has a "trivial" zero at m = 1 if and only if 
deg D is even. Thus 



1 degD even 
deg{D) odd 



^u, Xd) = {l-u) C*{u, Xd), A : 

where C*(u, xd) is a polynomial of even degree 

26 = degD - 1 - A 
satisfying the functional equation 

C*iu,XD) = {quyC*i — ,XD). 

qu 

In fact C*{u, Xd) is the Artin L-function associated to the unique nontrivial 
quadratic character of¥q{x){^/lJ{x)), see [Ml Propositions 17.7 and 14.6]. 
We write 



25 

C*{u,xd) = J2'^*d{^^^' 

13=0 

where A*jj(0) = 1, and the coefficients A*^{j3) satisfy 
(2.3) AUf3) = q^-'AU26-P). 

In particular the leading coefficient is A*^{26) = q^. 

2.4. For D monic, square-free, and of positive degree, the zeta function 
(11.11) of the hyperelhptic curve y"^ = D{x) is 

jO^*{u,Xd) 



Zn(u) 



[l-u){l-qu) 
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The Riemann Hypothesis, proved by Weil [121, asserts that all zeros of 
Zc{u), hence of C*{u, Xd), lie on the circle \u\ = l/y/q. Thus we may write 

'(^*{u, Xd) = det(/ - Uy/qQo) 
for a unitary 2S x 26 matrix Qd- 

2.5. By taking a logarithmic derivative of the identity 



desP\-l 



det(/ - u^Bd) = (1 - u)-^ - XDiP)u 



which comes from writing Xd) = (1 ~ ^) ^'^{u, Xd), we find 

A 



-trG"^ 



D ' qn/ 



(2.4) 

dog f=n 

2.6. Assume now that B is monic, of positive degree and not a perfect 
square. Then we have a bound for the character sum over primes: 



(2.5) 



E 

dcg P=n 

p prime 



deg 5 „,2 



n 



This is deduced by writing B = DC^ with D square-free, of positive degree, 
and then using the explicit formula (12.41) and the unitarity oi Qd (which is 
the Riemann Hypothesis). 



3. The hyperelliptic ensemble 7^2^ 



+1 



3.1. Averaging over 7^29+1- We denote by 7^^ the set of square-free monic 
polynomials of degree d in ¥q[x\. The cardinality of Tid is 



(l-i)g^ d>2 
d = 1 



as is seen by writing 



E 



q 



ds 



E 



i/r 



'^^^ f monic squarefree 

and using (12.11) . In particular for (7 > 1, 

m2,+i = {q- l)q'' . 



Us) 
C,(2s) 



TRACES OF HIGH POWERS 9 

We consider H2g+i as a probability space (ensemble) with the uniform 
probability measure, so that the expected value of any function F on 7^2g+i 
is defined as 

(3.1) {^)--=^^ E ^(^) 

We can pick out square-free polynomials by using the Mobius function fj, 
of ¥q[x] (as is done over the integers) via 



A2|Q 



1 Q square-free 
otherwise 



Thus we may write expected values as 

(3-2) (m)) = T-^ E E E ^i^M^'B) 

the sum over all monic A, B. 

3.2. Averaging quadratic characters. Suppose now that we are given a 
polynomial / G ^q[x] and apply (13.21) to the quadratic character XqU) = 
Then 



Hence 



where 



/fiWAy^r(f) gcd(A/) = i 

\ f J \ f J [0 otherwise 



2a+/3=2g+l deg_B=/3 ^ ^ 



a(/;«):= J] M^) • 



deg j4=« 
gcd(A,/)=l 

3.3. A sum of Mobius values. Suppose P is a prime of degree n, k > 1 
and a > 0. Set 

an{a) := cr{P'';a) = ^ /i(A) . 

deg A=a 
gcd{A,P'=)=l 
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Since the conditions gcd(y4, P'^) = 1 and gcd{A, P) = 1 are equivalent for 
a prime P and any k > 1, this quantity is independent of k; the notation 
anticipates that it depends only on the degree n of P, as is shown in: 

Lemma 4. i) For n = 1, 

cri(O) = 1, ai(Q;) = 1 — q for all a > 1 . 
a) If n > 2 then 

{1 a = mod n 
—q a = 1 modn . 
otherwise 

Proof. Since P is prime, 

degj4=a degj4=a deg A=a degj4i=a— n 

P\A 

Now /i(PAi) 7^ only when is coprime to P, in which case fi{PAi) = 
n{P)fi{Ai) = -fi{Ai). Hence 



degA=a degAi=a— n 

(P,Ai)=l 



that is 



deg yl=a 

on using 



1 


a 


= 


-Q 


a 


= 1 





a 


> 2 



A monic 

and (12. II) . For n > 2 we get (ii) while for n = 1 we find that cri(O) = 1 and 
for a > 1, 

(Ti(a) = (Ji(a -!) = ••• = = -q 
giving (i). □ 
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3.4. The probability that P \ Q. 
Lemma 5. Let P he a prime. Then 

Proof. Since P is prime, xq{P'^) = 1 unless P divides Q, that is setting 

p\f 

we have xq{P'^) = i^piQ) and thus by (13.21) 



Since P is prime, P f A^B if and only if P f A and P ^ B. Hence 

'^^ 0<a<g deg A=a,P\A deg B=2g+l-2a,P\B 

Writing m := degP, 

I 1, if m > /5 



and 

deg A=a,P\A 

is computed in Lemma HI Hence 



I I q \a=0 ^ 



29\ 



Moreover, inserting the values of o"m(a) given by Lemma H] gives 



E 



dm (a) 



1 



1 



(this is valid both for m = 1 and m > 2 !) and hence 
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as claimed. □ 

4. Double character sums 
We consider the double character sum 



degP=n deg B=f3 

p prime b monic 

We may express S'(/5, n) in terms of the coefficients Ap{[3) = J2dcg b=(3 Xp{B) 
of the L-function C{u, Xp) = ^2/3 ^p(/5)m^: 

SiP;n) = i-l)"^^- J2 MP), 

dog P=n 

which follows from the law of quadratic reciprocity |J£j : If A, B are monic 
then 

(^^^ = (_l)^degPdegB ^^^^ ^ ^_ ^ ^ 2_i dcg P deg ^^^j _ 

Since Ap{P) = for /? > degP, we find: 
Lemma 6. For n < (3 we have 

S{(3; n) = 0. 

4.1. Duality. 

Proposition 7. i) If n is odd and < (3 < n — 1 then 

(4.1) S{(3;n) =q^-'^S{n-l- (3]n) 
and 

71 — 1 

(4.2) S{n — l;n) = 7ig{n)q^2- ^ n odd . 
a) If n is even and 1 < P < n — 2 then 

-S{n-l-P;n) + {q-l) 5(j; n) 

j=o 

and 

(4.4) S'(n — 1; n) = — 7rg(n)g^~, n even . 
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Proof. Assume that n = degP is odd. Then C{u,xp) = j^*{u,xp), and so 
the coefficients Ap{P) = A*p{P) coincide. Therefore the functional equation 
in the form (12.31) imphes 

Ap{p) = Ap{n - 1 - P)qf^-'^ , n odd, < /? < n - 1 . 

Consequently we find that for n odd, 

S{P;n) = q'^-'^S{n-l- p;n), n odd, 0<p<n-l. 

In particular we have 

S{n — l;n) = S{0, n) = q~^Tiq{n), n odd . 

Next, assume that n = degP is even. Then C{u, xp) = (1 — u)C*{u, xp), 
which implies that the coefficients of C{u,xp) and C*{u,xp) satisfy 

Ap{(3) = A*p{[3)-A*p{f3-l), f3>l 

and 

(4.5) A*p{P) = Ap{P) + Ap{P - 1) + . . . + Ap{0) . 

Moreover 

Ap{0) = A*p{0), Ap{n-1) = -A*p{n-2) . 
In particular, since 

A*p{0) = l, A*p{n-2) = q— 

(see (12.31) ) we get 

n — 2 

Ap{n — 1) = —A*p{n — 2) = — n even 

so that 

S{n — l;n) = —T[q{n)q~^, n even . 
The functional equation (12.31) implies 

A*p{(3) = A*p{n - 2 - (3)q^-'^ , < /3 < n - 2 

and hence for 1 < (3 < n — 2 

Ap{(3) = A*p{(3) - A*p{(3 - 1) = A*p{n - 2 - (3)q^-'^ - A*p{n - 1 - /3)/-t 
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and inserting (14. 5p gives 

(n~l3~2 
_Apin-l-(3) + iq-l) J2 Mj) 

Summing over all primes P of degree n gives 

(n-/3-2 
_S{n-l-(3;n) + {q-l) ^ S{j;n) 

as claimed. □ 
4.2. An estimate for S{P]n). 
Lemma 8. Suppose (3 < n. Then 

(4.6) S{f5- n) = v^7T,{n)q^ + 0(^g5+^) 

where r]/^ = 1 for j3 even, and rj/^ = for [3 odd. 

Proof. We write 

«(ft«)= E E E E (I 

B=U degP=n ^ ^ Bj^n dcg P=n ^ 

deg B=/3 deg S=/3 

where the squares only occur when (3 is even. 

For B not a perfect square, we use the Riemann Hypothesis for curves in 
the form fl23D : 

E (f)«^.-. 

degP=n ^ ^ 

Hence summing over all nonsquare B of degree /?, of which there are at 
most g^, gives 



Bt^D deg P=n ^ ^ 
degP=/3 



E E 9 «V- 



Assume now that (3 is even. For B = C"^, we have P and -B are coprime 
since degC = (3 /2 < n = degP, and hence (^) = (^^j = +1 and so the 
squares, of which there are q^^"^^ contribute 7[g{n)q'^^'^. This proves (14.61) . □ 

By using duahty, (14.61) can be bootstrapped into an improved estimate 
when j3 is odd: 
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Proposition 9. If (3 is odd and (3 < n then 

(4.7) S{P; n) = -,^„vr,(n)g^-t + 0(g") . 

Proof. Assume n odd with (3 < n. Then by ( 14. ip for odd n, 

S{(3]n)=q^-'^S{n-l~ 13-n) 

and inserting the inequahty (14. 6 p with (3 replaced hy n — 1 — (3 (which is 
odd in this case) we get 

5(n-l-/3;n) <g^+("-i-^) 

hence 

5(/3;n)«g^-^gt+("-i-/5)«g" 

as claimed. 

Assume n even, with j3 < n. Using (14. 3 p and the bound (14. 6 p gives 

_Sin-l-P;n) + iq-l) S(r,n) 

j=0 

(n-l3-2 
-r]n-i--pTTq{n)q^^ + {q-l) ^ Tlj^q{n)q^ 
i=o 

n-l-P . 



O g/^-t ^ lq^+^ 



n 

j=0 

The remainder term is 0{q'^). For the main term, we note that n—l—(3 = 2L 
is even since /? is odd and n is even, and then we can write the sum as 

g^"t7r,(n) ^-g^ + (? " 1) 5Z j = -g''"^7r,(n) 
which is our claim. □ 

5. Proof of Theorem [T] 
The explicit formula (12.41) says that for n > 0, 



■0Q = -i E a(/)xq(/) 



qn/2 

deg f=n 
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the sum over all monic primes powers. We will separately treat the con- 
tributions Vn of primes, □„ of squares and EI„ of higher odd powers of 
primes: 

(5.1) tre^ = K + n„ + H„ . 

5.1. The contribution of squares. When n is even, we have a contri- 
bution to tr Oq coming from squares of prime powers (for odd n this term 
does not appear), which give 



^ dcg/i=f 

Since xq(^^) = or 1, we clearly have □„ < and 



^ degh=f 

by (12. 2p . Hence the contribution of squares is certainly bounded. 
Now for h = a, prime power, 

(5.2) {xQ{h')) = {xq{P')) = 1 - + 0{q-'^) . 

by Lemma [5l Thus, recalling that Sdeg/i=m^(^) ~ 9'™ (12. 2p . the contribu- 
tion of squares to the average is 

(5.3) 

-1 + ^^ E i^+ofe-)- 



q" 

degP| 2- 

In particular, we find that the contribution of squares to the average is 

Tl 

(□n) = -l+0(-^) + 0(g-^^) 
and thus if n ^ 3 logg g we get 

(□„) = -r/„(l + o(i)) . 

9 
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5.2. The contributions of primes. The contribution to tr0Q of primes 
is 

deg P=n 

Proposition 10. 

(5-4) (^n)=- .,_iL+n/2 E 

f3+2a=2g+l 
a,/3>0 

Moreover, if n > g then 

Tl 

(5.5) = {S{2g + 1; n) - g5(2^7 - 1; n)) . 







I) we 


have 




n 




l)(j2g+n/2 . 

do! 




n 



(^") = - r._lC2o+n/2 E E ^"(") E (I 



P 

/J=n /^+2a=2g+l degB=lj - 

— -, ^^r-, — TTT > 0"n(tt)S'(/9: n) 

/3+2a=2g+l 
«,/3>0 

which gives 05.41) . 

Now assume that n > g. Then (T„(q;) 7^ forces a = 0, 1 modn by 
Lemma |l](ii) and together with a < g < n we must have a = 0, 1. Hence in 
(15.41) the only nonzero terms are those with a = 0, 1 which gives (15. 5p . □ 

5.3. Bounding the contribution of primes. Assume first that n < g. 
In (15.41) . if S{P;n) 7^ then /3 < n by Lemma O For those, we use the 
bound \S{l3; n) \ -C ^q^~^^^'^ of Lemma [8] and hence 

(5.6) « ^ E « ^ aq-^' 

Q^9+ 2 ^-^ n 

since n < g, which vanishes as (7 — 00. 

For g < n < 2g, use (15. 5p . and note that S{2g ± 1; n) = by Lemma El 
Hence 

{Vn) = 0, g<n<2g. 
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5.3.1. The case n = 2g. We have S{2g + l;2g) = by Lemma El and 
^(2^ - 1; 2g) = -n{2g)q^ by g2D. Hence 

(Vn) = -J^Z^^ iS{2g + 1, 2g) - qS{2g - 1, 2g)) 

qTi{2g)q 2 



(g - l)q^9+9 
^ +0{gq-^). 



q-l 

5.3.2. The case 2g < n. Here we use (14.71) to find 

{Vn) = —, {S{2g + 1; n) - qS{2g - 1; n)) 



n 



(g - l)g29+t 



mi An) ^ , -n o„\ 
= ^-^^^ + ^ ') 

= r]^{l + 0{gq~'^))+0{nq'^-^3). 

The main term is asymptotic to and the remainder is o{l/g) provided 

2g < n < Ag - 5 log^ g . 

5.4. The contribution of higher prime powers. The contribution of 
odd powers of primes P'^, d > 1 odd, deg P'^ = n, is 

(72 — ^ d 

^ d\n degP=f 

3<d odd 

Since Xq{P'^) = Xq{P) ^ot d odd, the average is 



E ^ E E E (f ) 

''^ d\n dcgP=^2a+f3=2g+l degB=f3 ^ ^ 

3<d odd 



d|n 2«-|-/3=2c/-|-l 

3<d odd 
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In order that S{P;^) 7^ we need P < n/d. Thus using the bound 
SiP; 2) < Q^^^ of (M (recall that /3 < 2^ + 1 is odd here) gives 



^ d\n P<mm{n/d,2g+l) 



—+3 



n 



3<d odd 

E 

3<d odd 



Treating separately the cases n/3 < 2g and n/3 > 2g we see that we have 
in either case 



(5.7) 



5.5. Conclusion of the proof. We saw that 

(tre^) = (p„) + (n„) + (e„) 

with the individual terms giving 

r Oigq-^), 0<n<2g 
<Pn) = \ -^ + 0((7g-^), n = 2g 



2^^ 



degPlf 



and 



Putting these together gives Theorem [1] In particular 



3 log„ g <n<2g 



g-l' 



n = 2g 



> + 



0, 2c/<n<4^-81ogg^^ 
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6. The product of two traces 



Using the methods of this paper, on can also compute mean values of 
products of traces. For the product of two traces, the results can be stated 
as follows: 

Assume min(m, n) ^ log g and m + n < Ag — 100 logg g. Then 
i) If m = n then 



(6.i: 



(6.2) (tre^tre^)- < 



{n + r/n, n<g 
n + Vn + ^^, n = g 
n-l+?7„, g <n <2g -50\oggg 

ii) If m < n then for "generic" values of (m, n) we have 

^VmVn, m + n<2g 

VmVn - Vm+n, n<2g, 171 + u > 2g 

-Vm+n, ri> 2g, n-m<2g 

0, n — m > 2g 

while on "exceptional" lines we have 

{VmVn + -^, m + n = 2g 

VmVn-Vm+n + Vm-^, U = 2g 
-ipiVm+n, n-m = 2g 

The expected values for the symplectic group are O HJ [H [T2] : 
i) If m = n then 



(6.4) 



1 < n < g 



tiW'l^dU = <n-l + r]n, g + l<n<2g 



USp{29) 



2^7, 



n> 2g 



ii) If 1 < m < n 
(6.5) 



tr U"" tr W'dU = < 



USp{23) 



VmVn, m + n<2g 

VmVn - Vm+n, m<n<2g, m + n> 2g 

-r]m+n, n> 2g, n-m <2g 

0, n — m > 2g 



Comparing (16.41) . (16.51) . and (16.21) . (16.31) we find that if m = min(m, n) 3> 
logg g and m + n < ig — 100 log^ g then for "generic" values of (m, n), that 
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is if n, m 7^ 2g and |n ± m| 7^ 2g then 

(6.6) (tre^tre^) ~ /" iiU"'iiU''dU 

while on the hnes n, m = 25?, |n ± m| = 2g the difference between the 
averages over 7^29+1 and USp(25f) is bounded by 

I (trG^trO^) - /" trf/'^trf/WI < ^- + 0(1), ^^00. 
These results can be use to study the two- level density, compare [TTl [7] . 
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